A Path to Real-World Evidence in Critical Care Using Open-Source Data Harmonization Tools.

COVID-19 highlighted the need for use of real-world data (RWD) in critical care as a near real-time resource for clinical, research, and policy efforts. Analysis of RWD is gaining momentum and can generate important evidence for policy makers and regulators. Extracting high quality RWD from electronic health records (EHRs) requires sophisticated infrastructure and dedicated resources. We sought to customize freely available public tools, supporting all phases of data harmonization, from data quality assessments to de-identification procedures, and generation of robust, data science ready RWD from EHRs. These data are made available to clinicians and researchers through CURE ID, a free platform which facilitates access to case reports of challenging clinical cases and repurposed treatments hosted by the National Center for Advancing Translational Sciences/National Institutes of Health in partnership with the Food and Drug Administration. This commentary describes the partnership, rationale, process, use case, impact in critical care, and future directions for this collaborative effort.

Definitions matter: Heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis

Therapeutic efficacy in COVID-19 is dependent upon disease severity (treatment effect heterogeneity). Unfortunately, definitions of severity vary widely. This compromises the meta-analysis of randomised controlled trials (RCTs) and the therapeutic guidelines derived from them. The World Health Organisation ‘living' guidelines for the treatment of COVID-19 are based on a network meta-analysis (NMA) of published RCTs. We reviewed the 81 studies included in the WHO COVID-19 living NMA and compared their severity classifications with the severity classifications employed by the international COVID-NMA initiative. The two were concordant in only 35% (24/68) of trials. Of the RCTs evaluated, 69% (55/77) were considered by the WHO group to include patients with a range of severities (12 mild-moderate;3 mild-severe;18 mild-critical;5 moderate-severe;8 moderate-critical;10 severe-critical), but the distribution of disease severities within these groups usually could not be determined, and data on the duration of illness and/or oxygen saturation values were often missing. Where severity classifications were clear there was substantial overlap in mortality across trials in different severity strata. This imprecision in severity assessment compromises the validity of some therapeutic recommendations;notably extrapolation of "lack of therapeutic benefit” shown in hospitalised severely ill patients on respiratory support to ambulant mildly ill patients is not warranted. Both harmonised unambiguous definitions of severity and individual patient data (IPD) meta-analyses are needed to guide and improve therapeutic recommendations in COVID-19. Achieving this goal will require improved coordination of the main stakeholders developing treatment guidelines and medicine regulatory agencies. Open science, including prompt data sharing, should become the standard to allow IPD meta-analyses.

Definitions matter: Heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis.

Therapeutic efficacy in COVID-19 is dependent upon disease severity (treatment effect heterogeneity). Unfortunately, definitions of severity vary widely. This compromises the meta-analysis of randomised controlled trials (RCTs) and the therapeutic guidelines derived from them. The World Health Organisation 'living' guidelines for the treatment of COVID-19 are based on a network meta-analysis (NMA) of published RCTs. We reviewed the 81 studies included in the WHO COVID-19 living NMA and compared their severity classifications with the severity classifications employed by the international COVID-NMA initiative. The two were concordant in only 35% (24/68) of trials. Of the RCTs evaluated, 69% (55/77) were considered by the WHO group to include patients with a range of severities (12 mild-moderate; 3 mild-severe; 18 mild-critical; 5 moderate-severe; 8 moderate-critical; 10 severe-critical), but the distribution of disease severities within these groups usually could not be determined, and data on the duration of illness and/or oxygen saturation values were often missing. Where severity classifications were clear there was substantial overlap in mortality across trials in different severity strata. This imprecision in severity assessment compromises the validity of some therapeutic recommendations; notably extrapolation of "lack of therapeutic benefit" shown in hospitalised severely ill patients on respiratory support to ambulant mildly ill patients is not warranted. Both harmonised unambiguous definitions of severity and individual patient data (IPD) meta-analyses are needed to guide and improve therapeutic recommendations in COVID-19. Achieving this goal will require improved coordination of the main stakeholders developing treatment guidelines and medicine regulatory agencies. Open science, including prompt data sharing, should become the standard to allow IPD meta-analyses.

Preprints in times of COVID19: the time is ripe for agreeing on terminology and good practices.

Over recent years, the research community has been increasingly using preprint servers to share manuscripts that are not yet peer-reviewed. Even if it enables quick dissemination of research findings, this practice raises several challenges in publication ethics and integrity. In particular, preprints have become an important source of information for stakeholders interested in COVID19 research developments, including traditional media, social media, and policy makers. Despite caveats about their nature, many users can still confuse pre-prints with peer-reviewed manuscripts. If unconfirmed but already widely shared first-draft results later prove wrong or misinterpreted, it can be very difficult to "unlearn" what we thought was true. Complexity further increases if unconfirmed findings have been used to inform guidelines. To help achieve a balance between early access to research findings and its negative consequences, we formulated five recommendations: (a) consensus should be sought on a term clearer than 'pre-print', such as 'Unrefereed manuscript', "Manuscript awaiting peer review" or ''Non-reviewed manuscript"; (b) Caveats about unrefereed manuscripts should be prominent on their first page, and each page should include a red watermark stating 'Caution-Not Peer Reviewed'; (c) pre-print authors should certify that their manuscript will be submitted to a peer-review journal, and should regularly update the manuscript status; (d) high level consultations should be convened, to formulate clear principles and policies for the publication and dissemination of non-peer reviewed research results; (e) in the longer term, an international initiative to certify servers that comply with good practices could be envisaged.

Baseline results of a living systematic review for COVID-19 clinical trial registrations.

Background: Since the coronavirus disease 2019 (COVID-19) outbreak was first reported in December 2019, many independent trials have been planned that aim to answer similar questions. Tools allowing researchers to review studies already underway can facilitate collaboration, cooperation and harmonisation. The Infectious Diseases Data Observatory (IDDO) has undertaken a living systematic review (LSR) to provide an open, accessible and frequently updated resource summarising characteristics of COVID-19 study registrations. Methods: Review of all eligible trial records identified by systematic searches as of 3 April 2020 and initial synthesis of clinical study characteristics were conducted. In partnership with Exaptive, an open access, cloud-based knowledge graph has been created using the results.  Results: There were 728 study registrations which met eligibility criteria and were still active. Median (25 th, 75 th percentile) sample size was 130 (60, 400) for all studies and 134 (70, 300) for RCTs. Eight lower middle and low income countries were represented among the planned recruitment sites. Overall 109 pharmacological interventions or advanced therapy medicinal products covering 23 drug categories were studied. Majority (57%, 62/109) of them were planned only in one study arm, either alone or in combination with other interventions. There were 49 distinct combinations studied with 90% (44/49) of them administered in only one or two study arms. The data and interactive platform are available at https://iddo.cognitive.city/. Conclusions:  Baseline review highlighted that the majority of investigations in the first three months of the outbreak were small studies with unique treatment arms, likely to be unpowered to provide solid evidence.  The continued work of this LSR will allow a more dependable overview of interventions tested, predict the likely strength of evidence generated, allow fast and informative filtering of relevant trials for specific user groups and provide the rapid guidance needed by investigators and funders to avoid duplication of efforts.

The consequence of COVID-19 on the global supply of medical products: Why Indian generics matter for the world?

While the world is facing the urgency of the COVID-19 pandemic, policymakers must plan for the direct response to the outbreak while minimising its collateral impact. Maintaining the supply chain of pharmaceutical products is not only paramount to cover the immediate medical response but will be fundamental to reducing disruption of the healthcare delivery system, which requires constant medicines, diagnostic tools and vaccines for smooth functioning. In this equation, the role of the Indian pharmaceutical industry will not only be critical to meet the domestic need of over 1.3 billion inhabitants but will equally be important for the rest of the world, including wealthy economies. Preventing a significant disruption of the Indian pharmaceutical supply chain during the outbreak and preparing it for large scale production for COVID-19 therapeutic or preventive medical products will not only help India but will assist the global response to this outbreak.

A living systematic review protocol for COVID-19 clinical trial registrations.

Since the coronavirus disease 2019 (COVID-19) outbreak was identified in December 2019 in Wuhan, China, a strong response from the research community has been observed with the proliferation of independent clinical trials assessing diagnostic methods, therapeutic and prophylactic strategies. While there is no intervention for the prevention or treatment of COVID-19 with proven clinical efficacy to date, tools to distil the current research landscape by intervention, level of evidence and those studies likely powered to address future research questions is essential. This living systematic review aims to provide an open, accessible and frequently updated resource summarising the characteristics of COVID-19 clinical trial registrations. Weekly search updates of the WHO International Clinical Trials Registry Platform (ICTRP) and source registries will be conducted. Data extraction by two independent reviewers of trial characteristic variables including categorisation of trial design, geographic location, intervention type and targets, level of evidence and intervention adaptability to low resource settings will be completed. Descriptive and thematic synthesis will be conducted. A searchable and interactive visualisation of the results database will be created, and made openly available online. Weekly results from the continued search updates will be published and made available on the Infectious Diseases Data Observatory (IDDO) website ( COVID-19 website). This living systematic review will provide a useful resource of COVID-19 clinical trial registrations for researchers in a rapidly evolving context. In the future, this sustained review will allow prioritisation of research targets for individual patient data meta-analysis.